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ABSTRACT 



This study examined reasons why novice physicians, 



even after 6 years of medical education, are apparently unable to 
utilize a patient's contextual information (age, sex, profession, 
previous diseases, operations, and medical therapy) in the same 
accurate manner in disease diagnosis as their more experienced 
colleagues. Sixteen family physicians, averaging 14.7 years of 
experience, were compared with a group of 3 final year medical 
students and 13 physicians who graduated within 6 months prior to the 
experiment. One of two conditions were used for each subject: the 
first condition involved revealing first, the slide showing the 
complaint of the patient; the second, the patient's portrait; and 
third, the medical card. The second condition reversed the process. 
The subjects were asked, during pauses between case presentations, to 
state a most likely preliminary diagnosis, given the information 
presented. Contextual information recall and diagnostic accuracy of 
the two groups were analyzed, revealing that the experts generated 
more accurate diagnoses and recalled more contextual inf miation than 
the novices. It was concluded that physician experience a evaluating 
and diagnosing many types of illnesses was the contributing factor in 
the better use of patient information for diagnostic evaluations. 
Contains 10 references. (GLR) 
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Research in medical expertise has shown that already from the first moments in a consultation experi- 
enced physicians generate more accurate hypotheses about what is wrong with a patient than relatively in- 
experienced diagnosticians (Barrows, Norman, Neufeld, & Feightner, 1982; Elstein, Shulman, & Sprafka, 
1978). Many authors (e.g., Cutler, 1979; Hodgkin, 1978) have suggested that during this early diagnostic 
stage doctors use information about the patient's sex, age or previous disease history, which are important 
epidemiological aspects of disease. It is, however, unclear whether these authors' assumptions are only 
intuitive or empirically based. To our knowledge, only a few studies investigated the use of such 
information during early hypothesis generation by expert and novice physicians (Hobus, Schmidt, 
Boshuizen, & Patel, 1987; Hobus, Hofstra, Boshuizen, & Schmidt, 1988). Subjects in these experiments, 
medical experts and novice physicians, were presented with 18 short cases only consisting of a patient's 
complaint and information about the context of the patient (contextual information), such as age, sex, 
profession, previous diseases, operations and medical therapy. The experiments demonstrated that ex- 
perts' superior performance in generating accurate diagnostic hypotheses largely depended on the use of 
contextual information, whereas novices* hypotheses were not affected by this kind of information. Fur- 
thermore, experts recalled more case information, suggesting that they processed case information more 
intensively. So, the probably pre-sciennfic assumption about the role of contextual information is partly 
confirmed by the Hobus studies. 

The difference between expert and novice physicians is, however, remarkable. The textbooks that 
novice physicians have studied in medical school also address the epidemiological aspects of diseases like 
incidence rates, patient groups and risk factors. So why are novice physicians, even after six years of 
. medical education, apparently unable to utilize contextual information in the same fruitful way as their more 
experienced colleagues? One explanation might be tliat expert and novice knowledge structures differ. 
Eg., novice knowledge about epidemiological aspects of diseases might be only rudimentary. Another 
explanation might be that experts and novices have the same knowledge, but differing in accessibility. 



According to Schmidt and his colleagues (Schmidt, Norman, & Boshuizen, 1990; Schmidt, 
Boshuizen, & Hobus, 1988), medical expert knowledge representation is primarily script-based (see also 
Feltovich & Barrows, 1984). These scripts are the cognitive representations of diseases as they occur in 
humans. Each script contains the characteristics and context of a patient (described as 'Enabling Condi- 
tions') associated with the development of a malfunction in the organism ('Fault'). Enabling Conditions in 
this theory play a role similar to the contextual factors mentioned before. The Fault in turn leads to the 
signs and symptoms expressed by the patient ('Consequences'). For instance, being male, 47 years old, a 
heavy cigarette smoker and known with hypertension are enabling conditions for the development of my- 
ocardial atherosclerosis, a fault that leads to consequences such as dyspnoea, precordial pains or palpita- 
tions. Schmidt et al. (1988, 1990) assume that experts* illness scripts become more refined and 
differentiated over the years. Furthermore, medical experience results in script enrichment, especially with 
respect to enabling conditions. Hence, enabling conditions in the experts' scripts may be better tuned to the 
requirements of the early diagnostic stage. Another effect of clinical experience might be that the activation 
structure of illness scripts changes. At graduation, illness script might be only activated by a patient's 
complaint or signs and symptoms; later, through medical experience, this role can also be taken by 
contextual information. 

In order to study these two alternative explanations of the Hobus et al. (1987, 1988) results, an 
experiment was designed in which presentation order of contextual information and the patient's complaint 
was varied. If script enrichment and refinement is the major factor in developing expertise, then 
presentation order may not affect diagnostic accuracy and case recall of both expert and novice physicians. 
If, however, script accessibility changes, then experts and novices will be differentially affected by 
presentation order reversal. In both cases, experts are expected to perform better than novices. 

Method 

Subjects. Subjects were 32 physicians and graduate medical students. The expert group consisted of 
16 family physicians who on the average had 14.7 years of experience in primary health care (range 6-29). 
Average age was 43.6 years (range 35-59). The novices were three final year medical students who were 
about to graduate within three months and 13 physicians who graduated less than six months prior to the 
experiment. Their average age was 27.6 years (range 24-40). 



Material. The stimulus material consisted of 18 cases selected from a large bank of case-descriptions 
of real patients with known diagnoses. Case histories were selected such that contextual information could 
play a facilitative role in accurate diagnosis. The cases covered all organ systems. Each case was pre- 
sented on three slides. They displayed the portrait of the patient, his or her medical card, and the present- 
ing complaint. This complaint, consisting of one or two sentences, was an exact quotation of the patient's 
own words. For instance: "I have a cold fever for already two days long, doctor. I sometimes lie down 
shaking in my bed." The portrait 1 and medical card provided the contextual information of the complaint 
They contained information concerning the age and sex of the patient as well as his or her profession, past 
operations, medication use, hereditary disorders, smoking and drinking habits and notes on previous con- 
sultations. Slides were projected with a closed-box slide projector. Exposure time of each slide was fixed. 

Procedure. Subjects were randomly assigned to one of two conditions. In one condition the 1 8 
stimulus cases were presented as follows. The first slide showed the complaint of the patient, the second 
the portrait, and the third the medical card. In the other condition the presentation order of case information 
was reversed. First the patient's portrait was shown, then the medical card, and finally the complaint 
Between cases a black slide was projected for about 1 5 seconds. During that interval the subject was re- 
quested to state a most likely preliminary diagnosis, given the information presented In each condition the 
18 cases were presented in two series of nine each. Presentation order of these two series was varied sys- 
tematically over subjects in order to compensate for recency effects in the subjects' recall After a series 
was completed, each of the nine complaints presented, was read back to the subject, together with the 
tentative diagnosis the subject had generated. Subsequently, the experimenter asked the subject to recall 
whatever (s)he remembered from the case. All responses were given verbally and recorded on audio tape. 

Analysis. Diagnostic accuracy; Two independent raters compared the answers of the subjects with the 
actual diagnosis of the case. Interrater agreement was 92.1 %. 

Recall: A verbatim transcript was derived from the audio recordings. In order to measure the amount 
of contextual information recalled, the information on the patient charts was segmented into information 
units. An information unit was defined as a statement containing one singular fact or idea (Schmidt, 1982). 
For instance the line: 'Knee complaints; artrosis? contains two information units The complaint, *knee 



'The portrait was presented in order to give case presentations more "face'-validity. As an information source it was 
supposed to convey the sex and age of the patient only. Other characteristics of the facial appearance. like facial expression 
ERJjC ad hair color, were not of diagnostic importance. 



complaints', and the diagnosis, 'artrosis*. The verbatim transcripts of the subjects' responses was then 
compared with the segmented case-information. One point was given for each information unit recalled. 
In case of a summary, a distinction was made as to whether it referred to the content or to the frequency 
aspect of the summarized information. Two points were given when the summary referred to both 
frequency and content of a group of information units; one point was given when it referred to frequency or 
content only. Interrater agreement with this procedure was 91%. 

Group differences in diagnostic accuracy and recall of contextual information under the two case pre- 
sentation orders of the experiment were analyzed by means of two way analyses of variance. 



Results anc* Discussion 




Diagnostic accuracy and case recall by medical experts and novices under two case presentation orders. 

Figure 1 shows that experts generated more accurate diagnoses than novices: F ( 1 ,28) = 3 1 .45, p < 
.001 . Experts also recalled more than novices: F ( 1 ,28) = 5.28, p < .05. These results replicate the 
Hobus et al. (1987, 1988) findings described before. Presentation order did not affect the subjects' 
performances (diagnostic accuracy: F (1,28) - 1.46,/? = .24; recall: F (1,28) = .07,/? = .80). No 
interactions were found (diagnostic accuracy: F (1,28) = . 126, p = .73; recall: F(l,28) ■ 2.49, p = .13). 
These results are in agreement with the hypothesis that expert and novice illness scripts differ in richness 
and refmedness of the enabling conditions; they contradict the theory that expert and novice illness scripts 
have the same content, but differ in accessibility. 

It should be noted that these results hold for the role of enabling conditions in script development in 
general. However, they do not preclude that specific enabling conditions might play different roles. Eg., 
some enabling conditions might more easily trigger script activation than others, or some enabling condi- 



tions might especially play a role in further confirming or refuting already activated scripts. Furthermore, 
these roles might change through experience. In order to investigate this issue, a further analysis of the 
recall of classes of enabling conditions was done. 

ANOVA's showed that experts recalled more about the patients' sex (F (1,28) = 5.98,/? < .02), 
family history {F (1,28) * 9.29, p < .01), and medication (F(l,28) * 8.67, p < .01) than novice 
physicians. Interaction effects on recall of the patient's sex (F (1 ,28) = . 1 2, p - .73), family history (F 
(1,28) = .21, p = .65) and medication (F ( 1,28) = 2.87, p = .10) were not found. These results suggest 
that through experience illness scripts are primarily enriched with knowledge about the patient's sex, family 
history and medication. 

Furthermore, clear presentation order effects were found on the classes age (F ( 1,28) = 9.71, p = 
.004) and sex (F ( 1 ,28) = 8.82, p< .01). This information is better recalled in the complaint-context pre- 
sentation order. Also a presentation order effect was found on the category medication information (F 
( 1 ,28) = 5.88, p < .05). The latter kind of information was better recalled when contextual information 
was presented first. These differences suggest that there is no general way in which the activation of ill- 
ness scripts is triggered. Enabling conditions can both trigger illness script activation and confirm or refute 
the active script. The present results suggest that the patient's sex and age are good confirmatory or refut- 
ing information, but maybe too general to be efficient triggers themselves. For instance, a middle aged 
woman can potentially have such a large number of diseases that this information in itself is non-informa- 
tive. On the other hand, when it is known that the patient has pain in the lower abdomen, the same kind of 
information becomes of utmost importance in selecting the appropriate scripts. In such a case, certain dis- 
eases of the internal female genital tract become more likely, whereas youngster diseases like appendicitis, 
or male diseases like prostatitis can be discarded. The results suggest the opposite role for medication in- 
formation. In contrast to sex and age, medication provides more specific information with constant diag- 
nostic value. Information like 'Ibuprofen 400 mg 3 times daily' can trigger specific scripts related to the 
indications for such a prescription (e.g., artrosis and rheumatoid arthritis) or scripts related to the side-ef- 
fects of such drugs (duodenal ulcer and myeloproliferative disorders). The range of scripts that can be ac- 
tivated by a specific medicine is small. Medication information will, however, seldom significantly narrow 
down a broad range of possible scripts like sex and age can. As a result, attention towards and processing 
of medication information is different in the two presentation orders, due to the specific role of this en- 
abling conditions class in illness scripts. 



Finally, one interaction effect was found on a factor that did, however, not result in a main effect of 
level of expertise. When contextual information is given first, novice physicians recalled less information 
about previous diseases. Experts recalled more in this case than when the complaint was presented first (F 
(1,28) - 4.05, p < .05). From a medical point of view, information about previous diseases is very 
complex to interpret. Its value v> somewhere in between that of information about sex and age and infor- 
mation about medication. On the one hand, it can be almost non-informative, but also very selective (as 
soon as a set of illness scripts are activated). On the other hand, like information on medication use, it can 
be very concise and of stable diagnostic value. This finding suggests that with respect to previous diseases 
information the accessibility of the illness scripts changes as a result of experience. 

In conclusion, the development of expertise seems associated with increasing enrichment of illness 
scripts with enabling conditions resulting in a better diagnostic accuracy. The role of these enabling condi- 
tions in script activation or confirmation and refutation seems, however, not identical for every class of en- 
abling conditions. Finally, for one kind of enabling conditions the results suggest that script accessibility 
by this kind of information changes through increasing expertise. These results are, however, far from 
definitive. Systematic variation of the cases is needed in order to attain more empirical evidence. 



8 



References 



Barrows, H. S., Norman, G. R., Neufeld, V. R. & Feightner, J. W. (1982). The clinical reasoning 
of randomly selected physicians in general medical pncticc. Clinical d Investigative Medicine, 5, 49-55. 

Cutler, P. ( 1979). Problem solving in ciinical medicine. From data to diagnosis. Baltimore: The 
Williams & Wilkins Company. 

Elstein, A. S., Shulman, L. S., & Spraflca, S. A. (1978). Medical problem-solving: an analysis of 
clinical reasoning. Cambridge, MA: Harvard University Press. 

f 

Feltovich, P. J. & Barrows, H. S. (1984). Issues of generality in medical probiem solving. In: H. 

> 

G. Schmidt & M. L. De Voider. (Eds.), Tutorials in problem-based learning; A new direction in teaching 
the health professions. Assen: Van Gorcum. 

Hobus, P. P. M., Schmidt, H. G., Boshuizen, H. P. A., & Patel V. L. (1987). Contextual factors in 
the activation of first diagnostic hypotheses: Expert-novice differences. Medical Education, 21, 47 1 -476. 

Hobus, P. P. M., Hofstra, M. L, Boshuizen, H. P. A., & Schmidt, H. G. (1988). De context van 
de klacht als diagnosticum. (The context of the complaint as a diagnostic tool]. Hiusans en Wetenschap, 
31, 261-263. 

Hodgkin, K. (1978). Towards Earlier Diagnosis in Primary care . Edinburgh: Churchill Livinstone. 

Schmidt, H. G. (1982). Act i vatic van voorkennis, intrinsieke motivatie en de verwerking van tekst. 
I Prior knowledge activation, intrinsic motivation and text processing]. Doctoral Dissertation University of 
Limburg. Apeldoorn: Van Walraven. 

Schmidt, H. G., Boshuizen, H. P. A., & Hobus, P. P. M. (1988). Transitory stages in the 
development of medical expertise: The "intermediate effect" in clinical case representation studies. 
Proceedings of the 10th Annual Conference of the Cognitive Science Society, Montreal, Canada. 

Schmidt, H. G.. Norman, G. R., & Boshuizen, H. P. A. (1990). Cognitive research on clinical 
reasoning: Theory and implications. Academic Medicine , 65, 61 1-62 1. 



ERIC 



